Large data sets may exist in various sizes and organizational structures. With big data comprising data sets as large as ever, the volume of data collected incident to the increased popularity of online and electronic transactions continues to grow. For example, billions of records (also referred to as rows) and hundreds of thousands of columns worth of data may populate a single table. The large volume of data may be collected in a raw, unstructured, and undescriptive format in some instances. However, traditional relational databases may not be capable of sufficiently handling the size of the tables that big data creates. As a result, the massive amounts of data in big data sets may be stored in numerous different data storage formats in various locations to service diverse jobs, application parameters, and use case parameters.
Computing nodes in a distributed file system may be used to receive, process, and store data, and to run jobs to leverage the data. Computing nodes may also be logically partitioned to form various job queues configured to execute jobs. Jobs are typically assigned to queues based on initial configuration settings. For example, under a single queue setting, jobs may be executed sequentially regardless of complexity or queue resources needed. In a fair share methodology, jobs may be executed simultaneously and assigned queue resources such that all jobs get an equal share of the queue resources, regardless of complexity or queue resources needed.